1. Field of the Invention
This invention relates to set top video reception devices, and particularly, enhancing them to allow insertion of targeted indicia Into video.
2. Description of the Related Art
Electronic devices for inserting images into live video signals, such as described in U.S. Pat. No. 5,264,933 by Rosser, et al. and U.S. Pat. No. 5,491,517 by Kreitman et al., have been developed and used commercially for the purpose of inserting advertising and other indicia into video sequences, including live broadcasts of sporting events. These devices are capable of seamlessly and realistically incorporating logos or other indicia into the original video in real time, even as the original scene is zoomed, panned, or otherwise altered in size or perspective.
Live video insertion of indicia requires several steps. The event video must be recognized, tracked, and adjusted for the potential insert perspective and occluding objects prior to actual insertion. In the systems discussed in U.S. Pat. No. 5,264,933 by Rosser, et al. and U.S. Pat. No. 5,491,517 by Kreitman et al. it was assumed that the broadcaster would perform the complete process, including recognition, tracking, creating an occlusion mask, warping inserts to correctly match the current image, and correctly mixing the original video, warped insert and occlusion mask.
In U.S. Pat. No. 5,543,856 of Rosser, et al., a Live Video insertion System (LVIS) split into two functional parts is described, with an upstream, xe2x80x9cmasterxe2x80x9d part performing recognition and occlusion mask generation, and sending this information downstream, along with various control parameters, to a less computationally endowed downstream xe2x80x9cslavexe2x80x9d part, capable of warping inserts to correctly match the current image, and correctly mixing the original video, warped insert and occlusion mask.
A number of current trends in television, video, and computer technology make it feasible and economically likely that a xe2x80x9cslavexe2x80x9d LVIS unit will be included within future set-top units. One trend is toward broadcasters sending compressed video signals directly to the home. Compression is driven by a limited availability of broadcast bandwidth, especially satellite based broadcasts. In order to decompress the compressed video, users require a set-top device that has significant computing power and memory. These set top devices are required to run decompression algorithms in real time. Memory and computing power could also be utilized to make the set-top device act as a downstream xe2x80x9cslavexe2x80x9d part of an LVIS system.
A second trend is the decreasing price of memory and computing power thereby increasing a personal computer""s ability to process video bandwidths of information.
A third trend is the movement by telephone companies and other wire network providers to higher bandwidth networks. There is also the possibility that the World Wide Web, or some similar computer network, could become a means for large-scale data exchange or broadcast of high quality video information. Compressed video, still necessary to traverse networks with limited bandwidths, is decompressed by the personal computer receiving the data. The video processing power of the personal computer may be sufficient to also be utilized as the downstream xe2x80x9cslavexe2x80x9d section of an LVIS system.
Yet another trend is towards sending digital television signals directly to the home. This means that the television set itself will be a digital processor, potentially powerful enough to be programmed and used for the Image warping and the other processes required of a downstream slave unit of an LVIS system.
In all these scenarios, the significant point, as far as this invention is concerned, is that the set-top device, the last link of the video transmission chain, has significant computing power and memory. When this computing power and memory is sufficient for the viewer""s set-top device to act as the downstream or xe2x80x9cslavexe2x80x9d section of an LVIS system, a very interesting possibility arises-the possibility to target advertising within a mass medium. In particular, it makes narrow casting of advertising possible, particularly insertions, in television and other video transmissions.
To understand the benefits of narrow casting to television and video audiences, which is the subject of this application, it is useful to understand the concepts of targeting advertising.
The most pervasive, and precise, of existing methods of narrow-casting or target advertising is direct mail (aka junk-mail) which uses mail to deliver material to selected audiences. The starting point for direct mail is a database of addresses. These databases can also be cross-linked to so called profile factors, or personal information, pertaining to the residents at each address. These profile factors are typically age, income, family composition, number of children and their ages, type of automobile owned, dwelling type, zip code and various other demographic, psychographic and life-style information. The more of these profile factors the data base contains, the more useful it is for targeting the advertising. The data base is sorted by computer to generate a mailing list of candidates whose profile factors match an advertiser selected sub-set. The advertiser believes that clients whose profile factors fall within this selected sub set will be more responsive to buying the product the advertiser is selling, so that by mailing only to those people, the advertiser (or their client) can reach all of the audience who are highly predisposed to purchase their product, with the minimum of expense.
The use of these databases has three problems. The first is that they are only effective for mail. The more influential mass media, especially television, cannot be targeted, with anything like the same geographic precision because of the broadcast nature of the transmission. The second is the problem of trying to keep the data bases up to date. Typical sources of data used to compile such data bases, such as census information, professional licensed databases, credit card transactions, warranty cards, reverse directories and consumer surveys can be months, and more typically, years out of date, leading to considerable waste and to missing a substantial fraction of potential prospects. Even good data bases only guarantee 80-90% deliverabilityxe2x80x94i.e. 10-20% of the addresses are no longer valid. The third is the concern for privacy. The existence of such centralized data bases worries many people because of their potential misuse by agencies, including but not limited to government agencies, having authorized or unauthorized access to the data bases, and also their potential use by criminals for targeting theft, con schemes and other misdeeds.
The set-top downstream version of LVIS solves all three of these problems. First, it brings the power of direct marketing to video, in particular to the mass market medium of television. Moreover, it can do this in a way that avoids the need for centralized data bases, with their privacy and out-of-date concerns. The proposed targeting mechanism of this, application, Anonymous Target Profiling, effectively targets viewers profile factors without making them publicly available in a way that ensures profile factors are close to 100% current.
The invention comprises both a method and an apparatus to act as a Live Video Insertion System (LVIS), split into two functional parts, with an upstream, xe2x80x9cmasterxe2x80x9d part doing the recognition and occlusion mask generation, and sending this information downstream, along with various control parameters, to a less computationally endowed downstream xe2x80x9cslavexe2x80x9d part, capable of warping inserts to correctly match the current image, and correctly mixing the original video, warped insert and occlusion mask, where the downstream section is part of a set-top device in a viewer""s home.
Because of the location of the set-top device at the viewer""s television set, it becomes possible to narrow-cast video insertions to a single household, which may be a single person or even a particular TV set within a household. Narrow casting could be implemented as the television or video equivalent of direct mailing, in which a central, computer sorted data base is used to select viewers whose profile factors match an advertiser selected sub-set. For instance, the geographic location of set-top devices could be made extremely local by GPS type devices in the set-top device which may also double as theft protection mechanisms, or by phone numbers of attached modems, or postal codes, or by mailing addresses which are stored in the set top device, possibly as part of product warranty submissions. However, the availability of significant memory and computing power in the set-top device opens up a much more exciting possibility, which we term Anonymous Target Profiling (ATP).
Anonymous Target Profiling does not require a centralized database of all potential clients. Instead, there is a viewer usage recorder or monitor, located at the viewer location, and a viewer usage interpreter or key, supplied with the broadcast. The viewer usage recorder or monitor is a system which monitors television usage patterns and stores a continuously updated version of a usage profile. The set-top device is an ideal place to locate a viewer usage monitor. In a simple form, the viewer usage monitor would classify programs (or channels) and record a rolling viewing profile of viewing habits, including type of program watched, time of day and day of the week of viewing the program and duration of that viewing. More complex models of viewer usage may also include programs not watched, intensity of viewing (i.e. volume adjustments), surfing patterns (i.e. what video snippets arrest the attention of a channel surfer, even for a short time) and other more subtle aspects of viewer interaction with the medium.
A viewer usage interpreter is a key that translates the viewer usage profile into a set of profile factors associated with the viewing pattern. The viewer usage interpreter could be generated statistically by having a sample of households of known profile factors, who have their viewing habits monitored by a central system. By choosing the sample households scientifically so that each household in the television viewing population has a known chance of selection, the results obtained from the sample can be reliably projected to a larger television viewing audience. The sample size required for survey depends on the reliability needed. A moderate sample size is sufficient for most needs. For example, national polls, such as those conducted by the well known Gallup or Harris organizations, generally use samples of about 1,500 persons to reflect national attitudes and opinions to within an accuracy of xc2x14%. A sample of this size produces accurate estimates even for a country as large as the United-States with a population of over 250 million people.
In one usage of the invention, a broadcaster would establish a continuous survey of a few thousand households of known profile factors for each significant broadcast region. These surveys would be used to generate cross-correlations between viewer usage profiles and viewer profile factors. Advertisers wishing to have their advertising targeted to viewers with a particular sub-set of profile factors would be able to use the cross-correlations to translate their viewer profile requests into a viewer usage profiles request. The broadcaster would then send the required viewer usage profiles as part of the broadcast in for instance,.the vertical blanking interval (VBI) along with the advertisers insertion also in the VBI, over a number of fields, if necessary. At the viewer""s set-top, the device would see which insertion was linked to the local viewer usage profile, and insert appropriately.
For instance, on a widely watched event, such as the super bowl, a car company may chose to present different models, depending on the demographic or psychographic profile of the family, based on their viewing habits. As a simple example, a family with a viewing profile that includes significant viewing of young children""s programs is assumed to have children and may be shown advertisements for a mini-van, while a family with a profile that includes significant viewing of programs for out-door sports may be shown an advertisement for a sports utility vehicle made by the same company.
There could also be a xe2x80x9cwrite-inxe2x80x9d dimension to the viewing providing the viewer the opportunity to select extra specific profile factors. For instance, viewers who are looking for a car may add this fact to their viewer profile in order to deliberately solicit advertisements for cars. It may also be possible to specify price ranges and other relevant parameters.
In a further embodiment of the invention, insertions could appear as border advertisements surrounding, or partly surrounding one or more of the video windows, or could be a separate video window, which may change position, size, shape and orientation on the screen as a means of increasing the impact of the advertisement.
In another further embodiment of the invention, the set-top device could be used not only for in-programming advertising, as made possible by the LVIS system, but could use one or more secondary, possibly compressed, video channels as a source of alternate advertisements for showing in the conventional advertising breaks. These inter-program advertisements, usually coupled to events in the program, could also be at suitable breaks coupled to viewer action, such as when a viewer first turns the set on, switches channels, turns the set off, or alters viewing parameters, such as volume.
In a still further embodiment, in which the television is also connected to a computer network, such as the World Wide Web, the viewing profile could be extended to include a browsing profile, related to frequently visited web sites or other services requested. In addition, the advertising inserted could be web site addresses or other forms of links to further information, or further advertising, related to the product being advertised.
Much of the technology needed to implement the viewer usage monitor, necessary for Anonymous Target Profiling, could also be used to provide xe2x80x9csmartxe2x80x9d TV sets, which would favorably impact the economics of implementing the invention by allowing the set-top manufacturers or distributors to offset a substantial part of the cost of the set-top device to the end user. For example, a smart TV set, when turned on, would not just be on the channel it was on when it was turned off. Depending on the time and day of the week, it would turn on to the channel indicated most likely by the viewer usage profile, regardless of where it was when it was turned off. A smart television may also be used to provide user customized burn-ins, especially ones similar to those used by broadcasters to show baseball and football statistics. The extra channels and turners necessary for the network to offer alternate, full video to its advertisers, could also be used to have multiple windows, i.e., enhanced picture in a picture. Multiple windows would also enable the ability to turn on with predetermined setups more compelling. The warping necessary for the downstream, slave LVIS system, could be used to make one or more of these windows re-sizable, magnifyable (for people who wanted to examine some detail of the video) and even rotatable (for people who may want to lie down and have the video on its side as well). Writable digital video disks, or other high capacity, random access memory could be used by advertisers to store full motion video for insertion at the appropriate time. Such devices can also provide viewers with their own instant replay feature, automatically storing the last five or more minutes of what ever program was being watched. This feature would also make the magnification capability more compelling, especially for example to sports fans who may wish to go back and look at some aspect of play such as a ball landing close to a line in detail. Writable devices can also act as a scrap pad for grabbing bits of video they want to see later or show someone else; or as a more conventional video recorder.
These additional features may also be used as triggers for showing live or still video advertisements, either before or after the feature is used, or as a border advertisement during the use of the feature, or as a live video insertion on some recognized part of the video.
In addition to processing insertion information from a pattern recognition version of an LVIS system, the downstream, set-top part of the LVIS system could be taking insertion information from camera head sensors, including the types of systems developed for virtual studio systems, or it could be taking information from multi-user game applications.